Paediatric Early Warning Score to Detect Deterioration in Paediatric Patients at a Tertiary Care Hospital of a Developing Country: A Validation Study

By Wasif Ilyas Vohra, Ayaz Ur Rehman, Anosha Aslam, Salman Khan, Qalab Abbas

Affiliations

1. Department of Paediatrics and Child Health, The Aga Khan University Hospital, Karachi, Pakistan

doi: 10.29271/jcpsp.2024.10.1189

ABSTRACT
Objective: To test the validity of the paediatric early warning score (PEWS) and to identify the patients with deteriorating clinical conditions who were reviewed by a rapid response team (RRT) and were advised higher level of care.
Study Design: Observational study.
Place and Duration of the Study: Paediatric ward and high dependency unit (HDU) of The Aga Khan University Hospital (AKUH), Karachi, Pakistan, from January 2021 to March 2022.
Methodology: All children aged 1 month to 18 years, admitted with non-surgical diagnoses at AKUH and referred for an RRT consultation were identified by non-probability consecutive sampling. The bedside nurse assessed the PEWS and alerted the clinical team. The patient’s further course of action was decided based on the PEWS, detailed systemic examination, and laboratory workup. This aided in deciding the level of care (General ward, HDU, Paediatric Intensive Care Unit) required by the patients. Patients with length of stay >24 hours were included and those with did not resuscitate orders were excluded.
Results: Overall 10,032 patients were admitted to the ward and high dependency unit (HDU). Out of which, 323 (3.2%) patients had an RRT call and were included in the study. The median age of the study population was 3.15 years during the study period, and 30.3% were <1 year. System-wise admission diagnoses included respiratory (33.2%) and neurological diseases (16.1%). Median [interquartile range (IQR)] PEWS at RRT call was 2 (IQR, 1-2) on median admission day 2 (IQR, 1-3). The cumulative data at PEWS ≥3 demonstrated an area under the curve of 63% with a sensitivity of 90% and a specificity of 30%.
Conclusion: PEWS >3 can probably identify the patients at risk of deterioration with excellent sensitivity.

Key Words: Paediatrics, Rapid response team, Early warning score.

INTRODUCTION

Hospitalised children admitted to the wards can be at risk of deterioration due to disease evolution and infrequent monitoring, and if this goes unnoticed, it can lead to worse outcomes.¹ Before such deterioration, there is always a period of instability where certain abnormalities in clinical conditions can be picked.² To mitigate this, early warning scores have been developed to detect these changes.³ Early warning scores are considered an informative bedside tool. It was first used in 1990’s in the adult population. The British Infirmary Royal Report, published in 2001, was one of the leading point for the development of early warning scores in paediatrics.

This report identified significant failures in clinical care and leadership, including a lack of effective monitoring and response to signs of deterioration and recommended the development of a system to identify and respond to early signs of deterioration in hospitalised children.⁴

To improve the quality of care and vigilance in paediatric patients, the paediatric early warning score was introduced in 2005, which described a 3-item tool for detecting clinical deterioration in children.⁵ This scoring system was later referred to as the Brighton Paediatric Early Warning Score (B-PEWS) and was modified by Akre et al. in 2010.⁶ Since then, it has been used to identify patients at the risk of clinical deterioration and providing them optimal care by shifting to a higher level of care. The PEWS is based on monitoring three variables; cardiovascular, respiratory, and behaviour. A prospective study by Tucker et al. described the sensitivity and specificity of PEWS for detecting clinical deterioration that results in unplanned transfer to the paediatric intensive care unit (PICU). They reported a sensitivity of 90.2% and specificity of 74.4% at a score of 3 and concluded that PEWS could identify children who require transfer to PICU.⁷

Pakistan is a low-middle-income country (LMIC) with limited healthcare resources. Less skilled and overworked human resources and scarcity of medical equipment are some of the contributing factors of this fragile system. The progress is slow as Pakistan spends only 1.4% of its Gross Domestic Product (GDP) on health.⁸ Paediatric critical care services are not sufficient to cater all the patients in need.⁹ Therefore, it is important to have a simple scoring system, which can identify patients at risk of deterioration, in need of more clinical attention and ultimately prevent any sudden life-threatening condition. Furthermore, better utilisation of intensive care facilities can be made possible by identifying the patients in need of critical care. The objective of this study was to test the validity of the PEWS to identify the patients with deteriorating clinical conditions who were reviewed by an RRT and were advised higher level of care.

METHODOLOGY

This observational validation study was conducted at the Aga Khan University Hospital (AKUH), Karachi, Pakistan, between January 2021 and March 2022. Nurses were the frontline responders to alert the physicians on duty, and the role of the physician was to do detailed systemic examinations, decide the need for relevant laboratory workup, and along with on-duty nurse give an RRT call as required. Any member of the team (nurse and physician) responsible for children admitted to general paediatric floors could ask for help for a patient whom they thought was unwell, required escalation of care, was deteriorating, or if the patient’s family expressed concern. PEWS was implemented to recognise the deterioration in hospitalised children objectively. It was derived from 3 variables (neurology, cardiology, and respiratory); each variable scored from 0 to 3 as per clinical condition.⁵ The paediatric RRT calling criteria were acute changes in vital signs as per PEWS, acute change in Glasgow Coma Scale ≥3, oxygen saturation <90%, and if the nursing staff was worried. All nurses were trained for PEWS calculation which were randomly double-checked by the nurse team lead during implementation. For patients in general wards, this was performed with every vital sign checks after every 4-hours, while in the HDU, this was done every hour. All results were documented in the patient charts. Institutional protocol for escalation was followed if PEWS was above the normal limit or in an increasing trend. All children referred for an RRT call and with length of stay >24 hours were included in the study; those transferred to another hospital, who were with did not resuscitate orders (DNR), and those who left against medical advice were excluded.

Data were collected on a structured proforma via non-probability consecutive sampling and included PEWS and other details, including age, gender, presenting complaints, diagnoses, and level of care at admission which was collected from medical records. A sample size of 323 children who had an RRT was required to detect the sensitivity and specificity of PEWS.

Data of the patients’ maximum PEWS documented by the bedside nurse for each 24-hour interval of the patient stay were noted and used for statistical analysis. In case of review by RRT and escalation of care before 24 hours, the last documented PEWS was analysed. Statistical analysis was performed using STATA.

For comparative analysis, patients were stratified into two outcome groups: Patients with PEWS of less than 3 and patients with PEWS of 3 or more. Moreover, the baseline characteristics of patients on the basis of the requirement for escalation of care were also analysed. Normally distributed continuous variables were reported as mean ± SD, while non-normally distributed continuous variables were reported as median (IQR). Categorical variables were reported as frequencies and percentages. The Student’s t-test was performed to test the mean difference of potential factors with the outcome of interest, and the Rank-sum test was performed to assess the median difference, while the association was tested using the Chi-square test or Fisher’s exact test for categorical variables as applicable (level of significance being 0.05).

A 2x2 table was used to calculate sensitivity and specificity for PEWS by taking RRT as the gold standard. A receiver-operator characteristic (ROC) curve analysis was conducted to calculate and plot the corresponding sensitivity and specificity for all possible PEWS values, as predictors of the outcome of interest. The study was reviewed and approved by the Ethical Review Committee (ERC# 2021-5992-17082).

RESULTS

During the study period, 10,032 patients were admitted and 390 patients were enrolled in the study. Of these, 47 patients had missing data, and 20 patients did not meet the inclusion criteria.

The median [interquartile range (IQR)] was 3.15 year (7.8), and 202 (62.5%) were males (Table I). Presenting complaints included fever in 107 (33.2 %), respiratory distress in 107 (33.2%), and seizures in 26 (8.1 %) of the patients.

Admitting diagnoses were related to the respiratory system in 107 (33.2%) of patients, neurological in 52 (16.1%), oncology 31 (9.6%), cardiovascular 7 (2.2%), infectious disease 5 (1.6%), and gastroenterology 3 (1%), and the remaining 118 (36.6%) were classified as miscellaneous (Table I).

RRT call was given at PEWS of <3 in 261 (80.8%) patients and at ≥3 in 62 (19.1%) patients.

Table II shows the comparison between the two groups, the median day of admission at RRT call was 2.0 (1.0 - 2.0, IQR). Level of care after an RRT was escalated in 243 (75.2%) patients, including 183 / 261 (70%) in whom PEWS were <3 and 60 / 62 (96.7%) in whom PEWS were ≥3, the overall median length of stay was 10.1 ± 9.6 (IQR), it was 7 (5 - 10) in children who had PEWS <3 at RRT and ten days (6 - 15) in whom PEWS was ≥3 (Table II).

Figure 1 shows the receiver operating curve and Table III shows the sensitivity and specificity of different PEWS. Median PEWS was 2 (2) and 1 (1) in whom escalation of care was done and not done, respectively (Table III).

Table   I:   Patient   characteristics. 

Patient Characteristics	n (%)
	n = 323
Age of patient
<1	98 (30.3%)
1 to 3 year	59 (18.3%)
>3 to 7 year	71 (22.0%)
>7 years	95 (29.4%)
Gender of patient
Male	202 (62.5%)
Female	121 (37.5%)
Comorbid condition
Yes	112 (34.7%)
No	211 (65.3%)
Diagnoses
Neurological	52 (16.1%)
Respiratory	107 (33.1%)
Cardiovascular	7 (2.2%)
Gastrointestinal	3 (1 %)
Oncology	31 (9.6%)
Infectious disease	5 (1.5%)
Miscellaneous	118 (36.5%)
Presenting complaints
Fits	26 (8.1%)
Fever	107 (33.2%)
Respiratory distress	107 (33.2%)
Loose stool	10 (3.1%)
Trauma	8 (2.5%)
MISC	65 (20.2%)

Table  II:  Comparison  at  PEWS  <3  and  ≥3.

Baseline Characteristics (RRT)	PEWS AT RRT (< 3)	PEWS AT RRT (≥ 3)	p-value
	n = 261	n = 62
RRT given on day, mean ± SD	2.4 ± 1.6	3.2 ± 2.1	0.002
RRT given on day, median (IQR)	2.0 (1.0 - 3.0)	2.0 (2.0 - 5.0)	0.007
RRT given on day	n = 261	n = 62	0.001
1	100 (38.3%)	14 (22.6%)
2	63 (24.1%)	18 (29.0%)
3	47 (18.0%)	10 (16.1%)
4	25 (9.6%)	4 ( 6.5%)
5	6 (2.3%)	5 (8.1%)
6	7 (2.7%)	0 (0.0%)
7	13 (5.0%)	11 (17.7%)
Level of care escalated	n = 261	n = 62	<0.001
Yes	183 (70.1%)	60 (96.8%)
No	78 (29.9%)	2 (3.2%)
Shifted to, n = 321	n = 259	n = 62	<0.001
HDU	34 (13.1%)	0 (0.0%)
PICU	149 (57.5%)	60 (96.8%)
Not shifted	76 (29.3%)	2 (3.2%)
	n = 261	n = 62
Length of stay, mean ± SD	9.7 ± 9.6	12.1 ± 9.8	0.076
Length of stay, median (IQR)	7.0 (5.0 - 10.0)	10.0 (6.0 - 15.0)	0.006
Student’s t-test, rank-sum test, and Chi-squared test or Fisher’s exact test were applied for analysis of characteristics.

Table  III:  Sensitivity  and  specificity.

PEWS Score	Sensitivity %	Specificity %	ROC %
≥1	79	90	84
≥2	94	55	75
≥3	97	30	63
≥4	91	25	58

In the first two days of admission, there was deterioration in 195 (60%) patients.

In this study, the maximum PEWS ranged from 0 to 5; 80.8% of the scores ranged between 0 and 2. The median PEWS at RRT was 2, with an interquartile range of 1 to 2. Based on PEWS, 243 (75%) patients had an escalation of care; among these, 209 (65%) patients were shifted to the PICU for further management. Figure 1 illustrates a 63% area under the curve with a sensitivity of 97% and specificity of 30%.

Figure   1:   Receiver   operating   curve.

Furthermore, the p-value was <0.05 for the cardiovascular and respiratory components during the first three days and >0.05 for the behavioural component.

DISCUSSION

In this study, the effectiveness of PEWS was investigated to identify patients with clinical deterioration, keeping RRT call as the gold standard. It was found that the PEWS >3 was effective in predicting the clinical condition with a high sensitivity of 97%; however, specificity was low (30%).

The median age and gender of patients reviewed by RRT were comparable to previous data.¹⁰ The data depicted that admissions in the fourth quarter (October to December) were higher than in the first three quarters due to the number of respiratory cases. The significant diagnoses and presenting complaints were both related to the respiratory system. As the last quarter of the year has a winter season, more cases related to asthma exacerbation, pneumonia, and bronchiolitis were noted.

Respiratory distress was the chief complaint at the time of clinical deterioration, and this finding is consistent with the published literature.¹¹

The PEWS at the escalation of care were <3 in 80% of cases, this score is lower as compared to other studies; as seen in a study conducted by Dean et al., the score of 3 or more corresponded to the escalation of care to PICU.¹² The lower score was attributed to the lack of some facilities in ward and HDU, specifically less monitored beds in the ward and unavailability of High-flow nasal cannula therapy in HDU leading to the escalation of care at a score of  2.

The data depicted excellent sensitivity, but specificity is a concern compared to previous studies.^13,14 With a cut-off value set at 3, the sensitivity and specificity achieved were 97% and 30%, respectively. The specificity is problematic as this will result in an increased number of children being mistakenly categorised as prone to deterioration and a higher frequency of unnecessary notifications to physicians. The successful implementation of PEWS and RRT is related to human, social, and organisational factors such as healthcare professionals’ skills, organisational frameworks, social patterns, and local healthcare cultures.¹⁵ This data contributes to the existing literature supporting the effectiveness of PEWS in improving care and reinforces the efficacy of PEWS in enhancing healthcare for vulnerable patients. While there have been varying results concerning the influence of PEWS on mortality among paediatric patients hospitalised in well-resourced environments, research conducted in settings with limited resources has consistently demonstrated better patient outcomes.¹⁶ The studies available have demonstrated the potential to reduce mortality  while  also reducing resource utilisation in these  settings.^17-19

In terms of clinical deterioration, most patients deteriorated in the initial 48 hours of admission at a documented PEWS of 2. This study highlighted that a statistically significant number of patients assessed by RRT required escalation of care, and those who scored 3 or more were transferred to the PICU. This is a substantial observation as there were no code blue events during the study period due to the pre-emptive measures taken by the bedside nurse and suggestions by the RRT.²⁰ Although it is not the objective of current study, it is an interesting finding that the bulk of the deteriorating events occurred during the initial 48 hours. This could be due to disease-evolution as well as the decision for the disposition of patients to appropriate in-patient areas in the ward and the need for reassessment in the emergency room before disposition.

This study has several limitations. It was a single-centre study in a joint commission international accredited hospital with robust PICU services and RRT. Evaluation by RRT was considered an indicator for deterioration. As it is a teaching hospital, on some occasions, there could have been overuse or more reliance on the RRT, which is readily available and could contribute to a lower threshold for review by RRT. Moreover, lack of monitoring in the ward and unavailability of high-flow nasal cannula oxygen therapy in HDU also lead to early assessment and escalation of care to PICU. Furthermore, inter-rater reliability was not assessed due to the high attrition rate among registered nurses, and documentation requires uniformity for more standardised results. However, this has been previously studied and should not affect final results.

In LMIC, a multicentred study is required to assess the efficacy and feasibility of implementation in resource-variable settings and its effects on patient outcomes regarding length of stay, mortality, mechanical ventilation, and the requirement for other high-end modalities.

CONCLUSION

This study determined the reliability of PEWS to predict deterioration in hospitalised paediatric patients. It was noted that patients with scores of 3 and higher have an increased risk of requiring escalation of care and critical interventions. Further modifications in PEWS as per institutional policies and limitations can be done to mitigate false positives and negatives.   

ACKNOWLEDGEMENT:
The author is thankful to Shahiryar Khan for his exceptional contribution to this research project. His expertise in data analysis were instrumental in interpreting the complex data.  

ETHICAL  APPROVAL:
The study was reviewed and approved by the Ethical Review Committee of The Aga khan University Hospital, Karachi, Pakistan (ERC # 2021-5992-17082).

PATIENTS’  CONSENT:
The ERC determined that informed consent was not required due to the use of de-identified data. As such, the requirement for individual patient consent was waived.

COMPETING  INTEREST:
The  authors  declared  no  conflict  of  interest.

AUTHORS’  CONTRIBUTION:
WIV: Conceptualisation, methodology, writing and original draft preparation, and final revision.
AUR:  Formulation  of  the  idea,  data  curation,  editing,  and  investigation.
AA: Conceptualisation, resources, investigation of results, and final revision.
SK: Design, data curation, validation, methodology, and final revision.
QA: Conceptualisation, design of work, supervision, writing, reviewing, and editing.
All authors approved the final version of the manuscript to be published.

REFERENCES

1. Young KD, Seid JS. Paediatric cardiopulmonary resus-citation: A collective review. Ann Emerg Med 1999; 33(2):195-205. doi: 10.1016/s0196-0644(99)70394-x.
2. Harnden A, Mayon-White R, Mant D, Kelly D, Pearson G. Child deaths: Confidential enquiry into the role and quality of UK primary care. Br J Gen Pract 2009; 59(568): 819-24. doi: 10.3399/bjgp09X472520.
3. Lambert V, Matthews A, MacDonell R, Fitzsimonse J. Paediatric early warning systems for detecting and responding to clinical deterioration in children: A systematic review. BMJ Open 2017; 7(3). doi: 10.1136/ bmjopen-2016- 014497.
4. Dyer C. Bristol inquiry condemns hospital's "club culture". BMJ 2001; 323(7306):181. doi: 10.1136/bmj.323.7306.181.
5. Monaghan A. Detecting and managing deterioration in children. Pediatr Nurs 2005; 17(1):32. doi: 10.7748/paed 2005.02.17.1.32.c964.
6. Akre M, Finkelstein M, Erickson M, Liu M, Vanderbilt L, Billman G, et al. Sensitivity of the Paediatric early warning score to identify patient deterioration. Paediatrics 2010; 125(4): e763-9. doi: 10.1542/peds.2009- 0338.
7. Tucker KM, Brewer TL, Baker RB, Demeritt B, Vossmeyer MT. Prospective evaluation of a Paediatric inpatient early warning scoring system. J Spec Pediatr Nurs 2009; 14(2):79-85. doi: 10.1111/j.1744-6155.2008. 00178.x.
8. Health and Nutrition 2022. Available form: www.finance. gov.pk/survey/chapter_22/PES11-HEALTH.pdf?utm_medium =email&utm_source=transaction.
9. Abbas Q, Shahbaz FF, Hussain MZH, Ali Khan M, Shahbaz H, Atiq H, et al. Evaluation of the resources and inequities among paediatric critical care facilities in Pakistan. Pediatr Crit Care Med 2023; 24(12):e611-20. doi: 10. 1097/pcc.0000000000003285.
10. Gawronski O, Ferro F, Cecchetti C, Degli Atti MC, Dall'Oglio I, Tiozzo E, et al. Adherence to the bedside paediatric early warning system (Bedside PEWS) in a paediatric tertiary care hospital. BMC Health Serv Res 2021; 21(1):852. doi: 10.1186/s12913-021-06809-2.
11. Kowalski RL, Lee L, Spaeder MC, Moorman JR, Keim-Malpass J. Accuracy and monitoring of paediatric early warning score (PEWS) scores prior to emergent Paediatric intensive care unit (ICU) transfer: Retrospective analysis. JMIR Pediatr Parent 2021; 4(1): e25991. doi: 10.2196/ 25991.
12. Dean NP, Fenix JB, Spaeder M, Levin A. Evaluation of a paediatric early warning score across different subspe-cialty patients. Pediatr Crit Care Med 2017; 18(7):655-60. doi: 10.1097/PCC.0000000000001176.
13. Rosman SL, Karangwa V, Law M, Monuteaux MC, Daneau Briscoe C, McCall N. Provisional validation of a paediatric early warning score for resource-limited settings. Paediatrics 2019; 143(5):e20183657. doi: 10.1542/peds. 2018-3657.
14. Skaletzky SM, Raszynski A, Totapally BR. Validation of a modified paediatric early warning system score: A retro-spective case-control study. Clin Pediatr 2012; 51(5): 431-5. doi: 10.1177/0009922811430342. 
15. Shearer B, Marshall S, Buist MD, Finnigan M, Kitto  S, Hore T, et al. What stops hospital clinical staff from following protocols? An analysis of the incidence and factors behind the failure of bedside clinical staff to activate therapid response system in a multi-campus Australian metro-politan healthcare service. BMJ Qual Saf 2012; 21: 569-75.  
16. Chong SL, Liang Goh MS, Ong GY, Acworth J, Sultana R, Wen Yao SH, et al. Do paediatric early warning systems reduce mortality and critical deterioration events among children? A systematic review and meta-analysis. Resuscitation Plus 2022; 11:100262. doi: 10.1016/j.resplu. 2022.100262. 
17. Agulnik A, Mendez Aceituno A, Mora Robles LN, Forbes PW, Soberanis Vasquez DJ, Mack R, et al. Validation of a pediatric early warning system for hospitalized pediatric oncology patients in a resource‐limited setting. Cancer 2017; 123(24):4903-13. doi: 10.1002/cncr.30951.
18. Olson D, Preidis GA, Milazi R, Spinler JK, Lufesi N, Mwansambo C, et al. Task shifting an inpatient triage, assessment and treatment programme improves the quality of care for hospitalised Malawian children. Trop Med Int Health 2013; 18(7):879-86. doi: 10. 1111/tmi. 12114.
19. Chaiyakulsil C, Pandee U. Validation of pediatric early warning score in pediatric emergency department. Pediatr Int 2015; 57(4):694-8. doi: 10.1111/ped. 12595.
20. Tibballs J, Kinney S, Duke T, Oakley E, Hennessy M. Reduction of paediatric in-patient cardiac arrest and death with a medical emergency team: Preliminary results. Arch Dis Child 2005; 90(11):1148-52. doi: 10. 1136/adc.2004. 069401.Top of Form